In our earlier published patent publication, US20150227925, we described uses of printable conductive inks to convey information from an object to an electronic device through a capacitive sensor, such as a touch screen. We also described various techniques for conveying digital signals between a display screen and camera, and speaker and microphone, including using digital watermarking and variants of encoding digital payloads in image and audio carrier signals. Here, we detail further innovations in digital encoding and decoding of digital payloads in conductive ink structures.
As detailed in publications by Printechnologics Gmbh, T Ink and others, when an object printed with a pattern of conductive ink is placed on a capacitive sensor, such as a touch screen, the capacitive sensor senses the pattern defined by the ink and can respond accordingly. See, e.g., patent publications WO2012136817, WO2012117046, US20120306813, US20120125993, US20120306813 and US20110253789. Also, see also US Published Applications 20160098128, 20160100137, 20160067596, 20130284578, 20130115878, 20120125993, and U.S. Pat. Nos. 8,497,850, and 8,622,307. These documents further describe use of conductive ink as touch codes, and related technology is being commercialized under the TOUCHCODE brandname. Each of the above patent documents is hereby incorporated herein by reference in its entirety.
While such conductive ink technology has promise in conveying digital information, it has a number of limitations in terms of robustness, reliability, and security. In this document, we detail encoding and decoding technologies for conductive ink structures to provide improved robustness and reliability. We also describe encoding and decoding technologies that improve security, e.g., through encoding and decoding protocols and inter-dependencies with linked and layered payload signals in printed carriers.
Portions of this disclosure are described in terms of, e.g., encoded signals for product packaging (sometimes just referred to herein as “packaging” or “package”) and other objects. These encoding techniques can be used, e.g., to alter or transform how color and conductive inks are printed on various physical substrates. These techniques are further applied to create data carriers within conductive ink structures. The alterations or transformations preferably result in a printed design carrying machine readable indicia on a surface of a physical object.
Various forms of signal encoding (or “embedding”) include, e.g., “steganographic encoding” and “digital watermarking.” Digital watermarking is a process for transforming physical or electronic media to embed a machine-readable code (or “auxiliary data”) into the media. In some cases the media is modified such that the embedded code is obscured, yet may be detected through an automated detection process. Digital watermarking is often applied to electronic or physical objects such as printed objects, images, audio signals, and video signals. However, it may also be applied to other types of objects, including, e.g., product packaging, electronics such as circuit boards and CPUs, stickers, logos, product hang tags, line-art, software, multi-dimensional graphics models, and surface textures of such objects.
In this document, we use the terms “digital watermark” and “watermark” (and various forms thereof) interchangeably.
Auxiliary data embedding systems typically include two components: an encoder (or embedder) that embeds the auxiliary signal in a host image or object, and a decoder (or detector) that detects and reads the embedded auxiliary signal from the host image or object. The encoder may embed the auxiliary signal by altering or transforming a host image or object to carry the auxiliary data. The detection component analyzes a suspect image, object or signal to detect whether an auxiliary signal is present, and if so, extracts or reads information carried in it.
Several particular digital watermarking and auxiliary data embedding techniques have been developed. The reader is presumed to be familiar with the literature in this field. Particular techniques for embedding and detecting imperceptible digital watermarks are detailed in the assignee's patent documents including US Published Patent Application No. 2015-0156369; U.S. patent application Ser. No. 14/725,399, filed May 29, 2015, Ser. No. 14/724,729, filed May 28, 2015, and Ser. No. 14/842,575, filed Sep. 1, 2015; International Application No. PCT/US2015/44904, filed Aug. 12, 2015 (published as WO 2016025631 A1) and U.S. Pat. Nos. 7,054,461, 7,286,685, and 9,129,277. Related technology is detailed in Assignee's U.S. patent application Ser. No. 15/073,483, filed Mar. 17, 2016. Each of the patent documents mentioned in this paragraph are hereby incorporated herein by reference in its entirety, including all drawings and any appendices.
Digital encoding methods are used to encode digital payloads in image and conductive ink carriers. These carriers are applied to objects by various printing technologies, together in one or more ink formulations or in separate ink layers on an object. The image payload is extracted from an image sensed with image sensor, while the conductive ink payload is extracted from an image sensed with a capacitive sensor or like device for sensing the modulation in conductivity of the printed conductive ink elements. For example, a resistive sensor array or even a special purpose circuit layout may be used to read a conductive ink pattern. In the latter case, a special purpose reader circuit is placed in contact with the printed conductive ink pattern, forming a pattern that conveys an identifying signal by the connection made between the circuit reader and the printed pattern. In other reader embodiments, a touch screen display is integrated with a camera to enable sensing of both a color image and the conductive ink pattern when the screen is placed over the conductive ink pattern and color ink image.
The pattern printed using the conductive ink, therefor, carries both a signal or conductive identifier and simultaneously a digital watermark. The redundancy of the digital watermark that allows it to be effectively utilized with a variety of printed host image formats, can be inverted, allowing a “pure” digital watermark image, regardless of form, continuous tone (including approximated with multi-valued, 8 bit per pixel signal), sparse (sparse binary pattern), etc., to be altered, or otherwise augmented for design effect. This is motivated by aesthetic reasons or the in case of leveraging conductive ink, the functional requirements of creating electric circuits with a sensor array to achieve a desired reading effect.
The resulting design printed with a conductive ink faithfully represents the entirety of the digital watermark signal or portions thereof. In one configuration, the printed array of conductive ink elements contains only the synchronization elements of the aforementioned digital watermark. The message carrying elements are printed on a substrate, before or after printing the conductive ink pattern, using inks with colors in the visible band to carry a variable digital data payload, synchronization signal, or both variable payload and synchronization signal. This configuration allows machine readable informational structures to be layered, both to match the requirements of a manufacturing environment and the applicability of specific sensor technologies.
By way of example, consider a high value product, such as a pharmaceutical product, that requires both lot and item level serialization. The packaging of the product is created and processed as follows:
1. A digital payload is encoded in an image design. The image design comprises a specification for printing an array of conductive ink elements on a substrate. For example, this image design is printed on the enclosure of the package (e.g., a cap of the bottle, adhesive label over a container opening or the like) using a conductive ink, e.g., such as ink used in creating TOUCHCODE brand prints. The digital payload carries the lot number, manufacturing facility, etc. The conductive ink pattern forms a difficult to counterfeit feature as it requires access to the specialized ink.
2. The enclosure is read using a sensor in the factory to ensure the proper containers are filled. This sensor is a contact sensor or non-contact proximity sensor to sense the conductive ink structure, for example, by forming a circuit with resistive elements or capacitive sensing of the conductive ink array.
3. Subsequent to filling the container, a printer builds on the existing component of the digital watermark (e.g., a synchronization signal, binary pattern or code, or both). The printer builds on it by printing another image on the container that includes a digital watermark in a data channel of one or more color ink layers applied by the printer. The spectral channel or channels bearing this digital watermark signal may be manifested using process color inks (CMYK), spot color inks (e.g., see US 2015-0156369 A1), single color luminance modulation of the aforementioned inks, laser marking of a laser sensitizing additive (see our U.S. provisional application 62/505,771), and/or including of narrow band absorption dyes of pigments in an ink carrier or clear overprint that are readable in spectral band (e.g., a band around wavelength of 660 nm, typically used in barcode scanning equipment)(See our US Publication No. 2016-0275326). The overprint may be a varnish layer or a coating with colorant that matches another ink layer or substrate, so as not to make noticeable modifications to the package design. US provisional application 62/505,771 and US Publication No. 2016-0275326 are hereby incorporated by reference.
4. The printer may be an industrial ink jet, laser marking device, or other printer technology. This digital watermark applied in this layer of marking conveys an additional signal that conveys a unique identity of the particular container. For instance, this unique identify is conveyed via a digital code to uniquely identify or “serialize” that particular container.
5. Upon receipt of the container, the consumer uses her camera in the smartphone to read the full, serialized identity of it. In some instances, robust watermarks can be copied, so as an additional security check, the user is directed to place their smartphone screen on to the area of the conductive ink array (e.g., on the enclosure, such as the cap of the bottle or adhesive label forming a seal around the cap). The capacitive sensor of the smartphone screen senses the conductive ink array and generates a signal to confirm authenticity. The simplest case is mere detection of the ink array. A more sophisticated case is where the sensed pattern is processed as an image to decode a digital watermark, such as a synchronization signal, and optionally a digital payload demodulated from plural carrier signals. The synchronization signal provides a reference point (e.g., X, Y center or origin coordinates) and spatial orientation (rotation and scale) of a watermark signal tile.
6. The conductive ink utilized to print the digital watermark on the enclosure can also have additional properties that allow other marking technologies to be used to print the additional signal. One such example of this, is the use of a “receiver” in the ink that reacts to specific wavelengths of lasers. See our U.S. provisional application 62/505,771, incorporated above. This would allow very high-speed printing to occur on the caps of the bottles.
Further aspects, features and advantages will become even more apparent with reference to the following detailed description, claims and accompanying drawings.